Skip to content

Branch issue 8622: Remove python 2.7 - #9268

Closed
nsalwan wants to merge 4 commits into
apache:masterfrom
nsalwan:branch-issue-8622
Closed

Branch issue 8622: Remove python 2.7#9268
nsalwan wants to merge 4 commits into
apache:masterfrom
nsalwan:branch-issue-8622

Conversation

@nsalwan

@nsalwan nsalwan commented Jan 21, 2021

Copy link
Copy Markdown

Fixes #8622

Motivation

As per issue description : PIP-56 proposed we should upgrade our python version in the Docker image, we did that in the PR #7963.
But I still find some python2.7 usage in the pulsar repository.
We have fixed this issue.

Explain here the context, and why you're making that change. What is the problem you're trying to solve.
getting rid of python 2.7

Modifications

Describe the modifications you've done.
Updated the CMake files.

Verifying this change

  • Make sure that the change passes the CI checks.
    Yes, It does.

Documentation

  • If a feature is not applicable for documentation, explain why?
    It get rids of python 2.7 which is not supported by pulsar anyway.

@nsalwan nsalwan changed the title Branch issue 8622 Branch issue 8622: Remove python 2.7 Jan 21, 2021
@zymap

zymap commented Jan 22, 2021

Copy link
Copy Markdown
Member

@nsalwan Thanks for your contribution!

I saw it sill use python2.7 somewhere in the cpp tests:

creating 'dist/pulsar_client-2.8.0-cp27-cp27mu-linux_x86_64.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding '_pulsar.so'
adding 'pulsar/__init__.py'
adding 'pulsar/functions/__init__.py'
adding 'pulsar/functions/context.py'
adding 'pulsar/functions/function.py'
adding 'pulsar/functions/serde.py'
adding 'pulsar/schema/__init__.py'
adding 'pulsar/schema/definition.py'
adding 'pulsar/schema/schema.py'
adding 'pulsar_client-2.8.0.dist-info/METADATA'
adding 'pulsar_client-2.8.0.dist-info/WHEEL'
adding 'pulsar_client-2.8.0.dist-info/top_level.txt'
adding 'pulsar_client-2.8.0.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
---- Installing  Python Wheel file
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing ./dist/pulsar_client-2.8.0-cp27-cp27mu-linux_x86_64.whl
Collecting protobuf>=3.6.1
  Downloading https://files.pythonhosted.org/packages/41/c8/649a298f247305010c57ef2fffd8416686fbdb9e8d3754393ab4ee64f8cf/protobuf-3.14.0-cp27-cp27mu-manylinux1_x86_64.whl (1.0MB)
Collecting fastavro==0.24.0
  Downloading https://files.pythonhosted.org/packages/21/f1/8d5b9e88a37612bd8be6b20629b6f5e17e5196d98568bd17b60d00f0994f/fastavro-0.24.0-cp27-cp27mu-manylinux2010_x86_64.whl (1.1MB)
Collecting ratelimit
  Downloading https://files.pythonhosted.org/packages/ab/38/ff60c8fc9e002d50d48822cc5095deb8ebbc5f91a6b8fdd9731c87a147c9/ratelimit-2.2.1.tar.gz
Collecting certifi
  Downloading https://files.pythonhosted.org/packages/5e/a0/5f06e1e1d463903cf0c0eebeb751791119ed7a4b3737fdc9a77f1cdfb51f/certifi-2020.12.5-py2.py3-none-any.whl (147kB)
Collecting grpcio
  Downloading https://files.pythonhosted.org/packages/57/18/b093b4f51df5c16a5f37b34a067bdc8e966d107b41a01d6a0ac34d0658d4/grpcio-1.35.0-cp27-cp27mu-manylinux2010_x86_64.whl (3.9MB)
Collecting prometheus-client
  Downloading https://files.pythonhosted.org/packages/f4/7e/ef341c67ed43ad2e39633a35c28b77bc555f9572f4df4fee11c2b467db00/prometheus_client-0.9.0-py2.py3-none-any.whl (53kB)
Collecting six
  Downloading https://files.pythonhosted.org/packages/ee/ff/48bde5c0f013094d729fe4b0316ba2a24774b3ff1c52d924a8a4cb04078a/six-1.15.0-py2.py3-none-any.whl
Collecting apache-bookkeeper-client>=4.9.2
  Downloading https://files.pythonhosted.org/packages/82/f4/5a8d53f6d14cea2791c685ce2c715bb0b9fcbf832cda8406e583e22c75ec/apache_bookkeeper_client-4.12.1-py2.py3-none-any.whl (70kB)
Collecting enum34>=1.1.9; python_version < "3.4"
  Downloading https://files.pythonhosted.org/packages/6f/2c/a9386903ece2ea85e9807e0e062174dc26fdce8b05f216d00491be29fad5/enum34-1.1.10-py2-none-any.whl
Collecting pytz
  Downloading https://files.pythonhosted.org/packages/89/06/2c2d3034b4d6bf22f2a4ae546d16925898658a33b4400cfb7e2c1e2871a3/pytz-2020.5-py2.py3-none-any.whl (510kB)
Collecting futures>=2.2.0; python_version < "3.2"
  Downloading https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Collecting pymmh3>=0.0.5
  Downloading https://files.pythonhosted.org/packages/65/b0/738cb3e6237aa224f3bf48cd1d24f5ddf42ef7d03e6226012f9bdf7f665d/pymmh3-0.0.5-py2.py3-none-any.whl
Requirement already satisfied: setuptools>=34.0.0 in /usr/local/lib/python2.7/dist-packages (from apache-bookkeeper-client>=4.9.2->pulsar-client==2.8.0) (42.0.2)
Collecting requests<3.0.0dev,>=2.18.0
  Downloading https://files.pythonhosted.org/packages/29/c1/24814557f1d22c56d50280771a17307e6bf87b70727d975fd6b2ce6b014a/requests-2.25.1-py2.py3-none-any.whl (61kB)
Collecting urllib3<1.27,>=1.21.1
  Downloading https://files.pythonhosted.org/packages/f5/71/45d36a8df68f3ebb098d6861b2c017f3d094538c0fb98fa61d4dc43e69b9/urllib3-1.26.2-py2.py3-none-any.whl (136kB)
Collecting idna<3,>=2.5
  Downloading https://files.pythonhosted.org/packages/a2/38/928ddce2273eaa564f6f50de919327bf3a00f091b5baba8dfa9460f3a8a8/idna-2.10-py2.py3-none-any.whl (58kB)
Collecting chardet<5,>=3.0.2
  Downloading https://files.pythonhosted.org/packages/19/c7/fa589626997dd07bd87d9269342ccb74b1720384a4d739a1872bd84fbe68/chardet-4.0.0-py2.py3-none-any.whl (178kB)
Building wheels for collected packages: ratelimit
  Building wheel for ratelimit (setup.py): started
  Building wheel for ratelimit (setup.py): finished with status 'done'
  Created wheel for ratelimit: filename=ratelimit-2.2.1-cp27-none-any.whl size=5893 sha256=e5f98ddbf4de3c2898e2bdc214c23ae9d2eec3bcf8b0e4dd58b8e07e80b48972
  Stored in directory: /root/.cache/pip/wheels/05/d9/82/3c6044cf1a54aab9151612458446d9b17a38416869e1b1d9b8
Successfully built ratelimit
ERROR: apache-bookkeeper-client 4.12.1 has requirement grpcio<1.28,>=1.8.2, but you'll have grpcio 1.35.0 which is incompatible.
Installing collected packages: six, protobuf, pytz, fastavro, ratelimit, certifi, futures, enum34, grpcio, prometheus-client, pymmh3, urllib3, idna, chardet, requests, apache-bookkeeper-client, pulsar-client
WARNING: You are using pip version 19.3.1; however, version 20.3.3 is available.
Successfully installed apache-bookkeeper-client-4.12.1 certifi-2020.12.5 chardet-4.0.0 enum34-1.1.10 fastavro-0.24.0 futures-3.3.0 grpcio-1.35.0 idna-2.10 prometheus-client-0.9.0 protobuf-3.14.0 pulsar-client-2.8.0 pymmh3-0.0.5 pytz-2020.5 ratelimit-2.2.1 requests-2.25.1 six-1.15.0 urllib3-1.26.2
You should consider upgrading via the 'pip install --upgrade pip' command.
./run-unit-tests.sh: line 69:  2774 Aborted                 (core dumped) python pulsar_test.py

@BewareMyPower Do you have some advice for that?

@BewareMyPower

Copy link
Copy Markdown
Contributor

@zymap Maybe the python in run-unit-tests.sh could be changed to python3?

@zymap

zymap commented Jan 22, 2021

Copy link
Copy Markdown
Member

Yes, I think we need to remove all the python2 usage as possible as we can.

@BewareMyPower

Copy link
Copy Markdown
Contributor

The PythonLibs and Boost.Python dependency that are found by CMake are still Python2 related stuff.

-- Boost version: 1.58.0
-- Found the following Boost libraries:
--   program_options
--   regex
--   system
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython2.7.so (found version "2.7.12") 
-- PYTHON: 2.7.12
-- DETECTED Python 3
-- Boost version: 1.58.0
-- Found the following Boost libraries:
--   python-py27

The C++ tests CI environment is in image apachepulsar/pulsar-build:ubuntu-16.04, I think you can have a test in your local environment with the image.

$ ls /usr/lib/x86_64-linux-gnu/libboost_python*.so
/usr/lib/x86_64-linux-gnu/libboost_python-py27.so  /usr/lib/x86_64-linux-gnu/libboost_python.so
/usr/lib/x86_64-linux-gnu/libboost_python-py35.so
$ ls /usr/lib | grep python
python2.7
python3
python3.5

@codelipenghui

Copy link
Copy Markdown
Contributor

The pr had no activity for 30 days, mark with Stale label.

@github-actions

github-actions Bot commented Mar 4, 2022

Copy link
Copy Markdown

@nsalwan:Thanks for your contribution. For this PR, do we need to update docs?
(The PR template contains info about doc, which helps others know more about the changes. Can you provide doc-related info in this and future PR descriptions? Thanks)

@BewareMyPower

Copy link
Copy Markdown
Contributor

Close it by #15376

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Migrate the python2.7 to python3.7

4 participants